Bericht 245 A Conceptual Framework for High-level Vision FBI-HH-B-245/03

In this report we present essential elements of a conceptual framework for high-level vision (HLV). The scope of HLV is defined as scene interpretation above the level of object recognition. It is shown that models, on which such interpretations can be based, typically describe aggregates composed of meaningful parts, related to each other by temporal and spatial constraints. A frame-based representation is proposed which is based on techniques imported from configuration methodology. The hypothesise-and-test interpretation process is described for a table-laying example. It is shown that expectations are generated by partwhole reasoning. A temporal constraint net is proposed for the incremental evaluation of qualitative temporal constraints. A similar approach is sketched for spatial constraints which are represented by grid locations in a reference frame attached to an object. Scope of high-level vision (HLV) In this section we review developments in Computer Vision which contribute to a wider understanding of the vision task as compared to classical vision tasks such as recognizing or tracking single objects. These developments have to be taken into consideration when designing the conceptual framework for HLV in a cognitive agent as envisioned in the project CogVis. From human vision it is evident that what we see is interpreted in the light of diverse knowledge and of experiences about the world. The scope of this knowledge often termed common-sense knowledge can best be seen when we consider silent-movie watching as a Computer Vision task, for example, watching and understanding a film with Buster Keaton. If a vision system were to interpret the visual information of such a film in a depth comparable to humans, the system would have to resort to knowledge about typical (and atypical) behaviour of people, intentions and desires, events which may happen, everyday physics, the necessities of daily life etc. This is knowledge far beyond the visual appearance of single objects, and a vision system capable of silent-movie understanding clearly has to solve tasks beyond single-object recognition. As early as 1955 Computer Vision has been proposed as a task integrated in a cognitive context [Selfridge 55] and interacting with other cognitive processes. But Computer Vision research was in its infancy then, and a much narrower view of the vision task had to be pursued for several decades. The idea of integrating vision with other cognitive processes was actively investigated for the first time in the eighties in projects dealing with natural-language descriptions of imagery [vHahn et al. 80, Nagel 88, Neumann 89]. One of the important insights of this work was that qualitative descriptions had to be derived from geometric scene descriptions as an interface to language and symbolic reasoning.